\documentclass[a4paper,12pt]{article}
\usepackage[a4paper,left=28mm, right=28mm, top=28mm,bottom=28mm]{geometry}
\usepackage{fancyvrb}
\usepackage{color}
\usepackage[usenames,dvipsnames]{xcolor}

\definecolor{myBlue} {RGB}{0,0,155}
\definecolor{myBrown}{RGB}{60,40,0}
\definecolor{myGreen}{RGB}{0,40,0}

\usepackage[pdfpagelabels]{hyperref}
\hypersetup{
    %bookmarks=true,         % show bookmarks bar?
    unicode=false,          % non-Latin characters in Acrobat?s bookmarks
    pdftoolbar=true,        % show Acrobat?s toolbar?
    pdfmenubar=true,        % show Acrobat?s menu?
    pdffitwindow=false,     % window fit to page when opened
    pdfstartview={FitV},    % fits the width of the page to the window
    pdftitle={140.310 Practicum III Report},    % title
    pdfauthor={Hussain Al-Mutawa},     % author
    pdfsubject={Practical Work Office},   % subject of the document
    pdfcreator={PDFLatex},   % creator of the document
    pdfproducer={TexLive 2.9}, % producer of the document
    pdfkeywords={massey} {practical} {practicum} {engineering}  {software} 
     {jprobe} {testing} % list of keywords
    pdfnewwindow=true,      % links in new window
    colorlinks=false,       % false: boxed links; true: colored links
    linkcolor=myBrown,          % color of internal links
    citecolor=myGreen,        % color of links to bibliography
    filecolor=magenta,      % color of file links
    urlcolor=cyan           % color of external links
}

%\bibliographystyle{apasoft} % Choose Phys. Rev. style for bibliography
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}

\usepackage[printonlyused,withpage]{acronym}
%\usepackage[acronym,toc,shortcuts]{glossaries}
%\usepackage{amsmath}% perhaps you already use it

\usepackage{lmodern}
%\usepackage{natbib}
\usepackage[natbibapa]{apacite}
%\usepackage[hang,flushmargin]{footmisc} 
\bibpunct[, ]{(}{)}{,}{a}{}{;}
\setcitestyle{aysep={,},yysep={;}}

%\usepackage[T1]{fontenc}
%\usepackage{times}

\usepackage[T1]{fontenc}
%\usepackage[scaled=0.9]{helvet}
%\renewcommand*\familydefault{\sfdefault}

\newcommand{\Section}[1]{\clearpage \section{#1}}

%\usepackage{verbatim}
%\renewcommand{\comment}[1]{}

\pretolerance=150
\setlength{\emergencystretch}{3em}


\newcommand{\hilight}[1]{\colorbox{yellow}{#1}}
\newcommand{\nl}{\vspace{1em}}
\newcommand{\W}[1]{\hilight{\LARGE \textcolor{red}{#1}}}
\newcommand{\p}{\relax \par \tolerance=150 \noindent}
\newcommand{\h}[1]{\p \textbf{\Large{#1}} \vspace{1em} \relax}



\begin{document}

%%%%%%%%%%% TITLE PAGE
	\begin{titlepage}
		\begin{center}
		%\vspace*{1cm}
		\large
		{\color{myBlue}\bfseries \Huge Automated Software\\[4mm] Test Generation with jProbe}\\
		\vspace{3cm}
		%\LARGE
		{Practicum Project Report\\[0.5\baselineskip]
		by\\[0.5\baselineskip]
		Hussain Al-Mutawa\\
		{\small {B.E. Software Engineering (Honours)}}}\\
		%\comment{test comment}
		\vspace{3cm}
		{\tt \today}\\ %%Date - better you write it yourself.
		\vspace{1em}
		{Supervisor:\\
		A/Prof. Jens Dietrich}\\
		\vspace{1em}
		{Mentor:\\
		A/Prof. Eva Heinrich}\\
		\vspace{1em}
		\includegraphics[width=0.25\textwidth]{logo}\\[5mm]
		{\small Practical Work Office}
		\end{center}
	\end{titlepage}

\setcounter{topnumber}{0}
\pagenumbering{roman}
%%%%%%% Summary
\h{Summary}
\p This report was completed as a partial fulfilment of the requirements for
the third practicum course for the bachelor of engineering with honours degree
program at Massey University. The practicum took place at Massey University,
\ac{SEAT} during the summer of 2011 in the period from the 16th of  November
2011 until the 28th of February 2012. The report is a summary for the knowledge
gained about the employer and the own tasks performed during the training
program. The first part of the report provides the research done about the 
employer and is presented in the following points:
		
		\begin{itemize}
		\item History and background
		\item Objectives goals, vision and values
		\item Financial aspects
		\item Management structure
		\item Training philosophy / Development and Industrial relations
		\item The local market and competition
		\end{itemize}

\p The second part of the report describes the nature of the job tasks and
illustrates the activities taken during the analysis and design phases to
develop the main training project.\nl
\p The last part of the report provides a personal evaluation, some
conclusions about the training program, comments about business processes in
relation to the project, and presents some recommendations to improve
employer's products and services.

\clearpage	

%%%%%%% ACKNOLEDGMENT
\h{Acknowledgment}
\p I extend my profound gratitude to my supervisors (Jens Dietrich and Sayed
Ali Shah) for all the help and guidance provided during the course of this
project.\nl
	
\newpage
\tableofcontents
\newpage
\listoffigures
\listoftables
\vspace{1em}
\h{Glossary and Acronyms}	
%%%%%%%%%%%%%%%%%%%
\begin{acronym}[CodeProAnalytix]
\acro{CodeProAnalytix}	{Java software testing tool for Eclipse developers}
\acro{GUI}				{Graphical User Interface}
\acro{IDE}				{Integrated Development Environment}
\acro{I/O}				{Input/Output}
\acro{CVS}				{Source Version Control}
\acro{Emma}				{Java Code Coverage Inspection Tool}
\acro{JPF}				{Java PathFinder}
\acro{JUnit}			{Java Unit Test Tool}
\acro{NASA}				{National Aeronautics and Space Administration}
\acro{OO}				{Object-Oriented}
\acro{PP}				{Pair Programming}
\acro{PUB}				{Publication}
\acro{RegEx}			{Regular Expression}
\acro{SEAT}				{School of Engineering and Advanced Technology}
\acro{SP}				{Solo Programming}
\acro{SCM}				{Software Configuration Management}
\acro{SVN}				{sub-versioning}
\acro{TDD}				{Test-Driven Development}
\acro{TF}				{Test-First Programming}
\acro{TL}				{Test-Last Programming}
\acro{XML}				{eXtensible Markup Language}
\acro{XP}				{eXtreme Programming}
\end{acronym}%[XXXXXXXXXX]
%%%%%%%%%%%%%%%%%%%
\clearpage

\pagenumbering{arabic}
\setcounter{topnumber}{0}

%%%%%%% INTRODUCTION
\Section{Introduction}
\p During the summer holiday of 2011, approximately 300 hours of full time
work was done as outlined in the requirements of the bachelor of engineering
practical work guidelines and regulations. The work assignment has been done
mainly in software testing. During the work period, constant communication
between the client, supervisors and technical support staff was maintained on
a daily basis. The end product (jProbe) was completed, delivered and used by
the employer. In the end, the valuable skills learned during the process have
made it an excellent work period.\nl

\p The first part of the report discusses the organization's background, 
main products and services, targeted markets, the technologies and processes 
used to both plan and manage the company, a description about the work done 
and the role held in the company.\nl

\p The second part discusses the tasks accomplished and the insights gained
throughout the training program. The software developed and the development
process followed.\nl

\Section{The Employer}
\p The \ac{SEAT} at Massey University
in Palmerston North was the employer who provided this training opportunity. 
The main task assigned was to work as a research assistant with Sayed Ali Shah 
to generate test cases for the Qualitas Corpus \footnote{The Qualitas 
Corpus is a curated collection of software systems intended to be used for 
empirical studies of code artefacts. The primary goal is to provide a resource 
that supports reproducible studies of software. The current release of the 
Corpus contains open-source Java software systems, often multiple versions,
\citep{AT10}
}  
projects.

\subsection{History and Background}
\p Massey University is one of the eight governmental higher education 
institutions in New Zealand. It was founded in 1927 as Massey Agricultural 
College and named after former Prime Minister William Ferguson Massey. Massey 
became a university in 1964. \citep{UNZ09}.

\subsection{Goals and Objectives}
\p Excerpt from: \citep{MC09}
\begin{quote}
\p Massey University is committed to meeting the needs of New Zealand and New 
Zealanders, enhancing access to university study for diverse populations, 
preparing students for life-long learning, and meeting international standards 
of excellence in research and teaching. Massey University is an integrated 
multi-campus institution of higher learning that creates new knowledge and 
understanding; synthesises, applies and disseminates knowledge; develops 
advanced learning and scholarly abilities for a national and international 
student body; and promotes free and rational inquiry. We offer high-quality 
learning experiences that empower people and their communities to prosper in 
an increasingly knowledge-dependent and technologically advanced world.\nl

\p Massey University is driven by a spirit of community relevance and 
engagement, while maintaining intellectual independence. We will use our 
multi-campus structure to meet the needs of our constituent regional 
communities, while our flexible delivery and distance (extramural) education 
capabilities give a national and international reach to our educational 
programmes. \nl

\p Massey University recognises and respects the significance of mana whenua 
within its regions and the range of Maori organisations contributing to Maori 
development and advancement. We have demonstrated our commitment to Maori 
development by providing Maori academic leadership, research opportunities and 
educational qualifications that assist in the achievement of Maori
aspirations.\nl

\p Our integrated academic structures and organisational arrangements enable
and support interdisciplinary and cross-disciplinary research and academic
programmes. We pride ourselves on the relevance of our programmes; on our
openness to students of diverse backgrounds spanning age, geographic location,
educational background, ethnicity and culture; on the support we provide for our
students; and on the relationship we have built with our alumni.
\end{quote}


\subsection{Mission Statement}
\begin{figure}[htbp]
  \centering
  \fbox{
    \includegraphics[width=0.5\textwidth]{mission}
  }
  \caption{Massey mission statement}
  \label{fig:mission}
\end{figure}
\p Massey University is committed towards creating leaders and transforming 
businesses. Creating Leaders means that our top graduates should excel in 
positions of responsibility or influence in an enterprise that has global reach.
Transforming Business means that we will effect change through the activities 
of our graduates and we will also show our New Zealand business partners (both 
profit and not-for-profit) how our research can benefit them, \citep{MM11}.

\subsection{Organizational Structure}
\p Massey University's senior leadership team is responsible for the management 
of the university and is led by the Vice-Chancellor, Steve Maharey, 
(Figure \ref{fig:ochart}).
\begin{figure}[htbp]
  \centering
  \fbox{
    \includegraphics[width=0.9\textwidth]{ochart}
  }
  \caption{Massey University organization structure}
  \label{fig:ochart}
\end{figure}


\subsection{Financial Aspects}
\p Massey University is the largest distance learning provider in New Zealand, 
and has about 36,000 students, and about half of them are extramural 
	\footnote{
		Extramural is the term used at Massey University for distant 
		learning students
	}
and one in every four students is a postgraduate. 
Massey has about 3000 staff members and its assets are valued at over 
NZ\$1 billion, including 2000ha of farms used for research and teaching, 
and total annual revenue of NZ\$350 million, \citep{UNZ09}

\subsection{Development and Industrial Relations}
\p The University supports academic or general staff undertaking further 
academic study. In some instances, assistance with some time off work and fees 
concession is available. Details can be found in the provisions of the 
employment agreements and at staff study, \citep{MD12}.

\subsection{Local Market and Competition}
\begin{figure}[!ht]%[htbp]
  \centering
  \fbox{
    \includegraphics[width=0.75\textwidth]{arwu}
  }
  \caption{Overall ranking of New Zealand universities}
  \label{fig:arwu}
\end{figure}

\p Massey University is ranked in top 401-500 university worldwide and in top 
3-5 universities nationwide, \citep{ARWU} and \citep{SM10}. 
Figure \ref{fig:arwu} shows that NZ universities ranking where Massey University
ranking went slightly down during the last two years as compared to the average
ranking. \citet{SM10} suggests that the reason for Massey University's drop in 
performance appears to be a result of a fall in their relative \ac{PUB} 
\footnote{
Total number of papers indexed in Science Citation Index-Expanded and Social 
Science Citation Index in 2009. Only publications of 'Article' and 'Proceedings 
Paper' types are considered. When calculating the total number of papers of an 
institution, a special weight of two was introduced for papers indexed in Social
Science Citation Index, \citep{SM10}. } measure.


\Section{Task Analysis}

\subsection{Task Specification and Execution}
\p The main task assigned was to generate test cases for the corpus data set. 
There are about 100 open source projects that are used in an experiment aimed 
to improve the maintainability of software components. The experiment is applied
to such projects and does the following steps:
\begin{itemize}
	\item	performs dependency analysis
	\item	runs test cases and keeps record of test results, i.e., 
			{\tt Test(before)}
	\item	detects circular dependencies
	\item	applies refactoring to remove circular dependencies
	\item	runs test cases and keeps record of test results, i.e., 
			{\tt Test(after)}
	\item	the refactoring tool shall not introduce new bugs, and therefore, \\
			{\tt Test(before)=Test(after)}
\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  QUALITAS LIST OF SYSTEMS TABLE
\begin{table}[!htb]
\begin{center}
\begin{tabular}{ |l l l l l| }
\hline  

ant	&	findbugs	&	jboss	&	jtopen	&	quilt	\\
antlr	&	fitjava	&	jchempaint	&	jung	&	roller	\\
aoi	&	fitlibraryforfitnesse	&	jedit	&	junit	&	rssowl	\\
argouml	&	freecol	&	jena	&	log4j	&	sablecc	\\
aspectj	&	freecs	&	jext	&	lucene	&	sandmark	\\
axion	&	galleon	&	jfreechart	&	marauroa	&	springframework	\\
azureus	&	ganttproject	&	jgraph	&	maven	&	squirrel\_sql	\\
c\_jdbc	&	gt2	&	jgraphpad	&	megamek	&	struts	\\
castor	&	heritrix	&	jgrapht	&	mvnforum	&	sunflow	\\
cayenne	&	hibernate	&	jgroups	&	myfaces\_core	&	tapestry	\\
checkstyle	&	hsqldb	&	jhotdraw	&	nakedobjects	&	tomcat	\\
cobertura	&	htmlunit	&	jmeter	&	nekohtml	&	trove	\\
colt	&	informa	&	jmoney	&	netbeans	&	velocity	\\
columba	&	ireport	&	joggplayer	&	openjms	&	webmail	\\
compiere	&	itext	&	jparse	&	oscache	&	weka	\\
derby	&	ivatagroupware	&	jpf	&	picocontainer	&	xalan	\\
displaytag	&	jFin\_DateMath	&	jrat	&	pmd	&	xerces	\\
drawswf	&	jag	&	jre	&	poi	&	xmojo	\\
drjava	&	james	&	jrefactory	&	pooka	&		\\
eclipse\_SDK	&	jasml	&	jruby	&	proguard	&		\\
emma	&	jasperreports	&	jsXe	&	quartz	&		\\
exoportal	&	javacc	&	jspwiki	&	quickserver	&		\\

\hline
\end{tabular}
\end{center}
\caption{Qualitas corpus systems}
\label{table:qualitas}
\end{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Software Testing: Background}
\p ``Testing is a set of activities that can be planned in advanced and 
conducted systematically'', \citep{AG09}. \citet{DJ72} claimed that ``program
testing can be used to show the presence
of bugs, but never their absence'' to persuade us that a testing approach is not
acceptable.\nl

\p The size of Qualitas Corpus projects varies and some are extremely large. 
Therefore, manually written test cases may require a very long time. 
Table \ref{table:qualitas} lists the projects (systems) used in the Qualitas
collection of software systems.
\citet{SV07} suggests that testing is considered as one of the 
core developmental steps which is used to validate the correctness and quality 
of any software. Indeed, testing a product is carried out to 
check if it meets the requirements specified by the end-user. Therefore, 
validation and verification (V \& V) starts with requirements reviews and 
continuous throughout the design process.\nl

\p \citet{SV07} defines validation as checking if the product built 
is the correct product (meets expectations), and defines verification as 
checking if the product is built correctly (conforms to specifications). Testing
is estimated to have about half the cost of the development of a successful 
software, \citep{WV04}. \citet{EG07} claim that the estimated loss of resources 
caused by software packages which are not well tested to be about \$60 million 
per year in the United States. \citet{PR05} and \citet{EG07} suggest out that 
the cost of failure is exponentially related to the size of a software. Software
testing is aimed to be a prevention measure of unexpected faults and bugs in
the future.

\subsubsection{Software Testing Goals, Objectives and Principles}
\citet{SV07} suggests that the software testing process should have two distinct
goals:
\begin{enumerate}
	\item To demonstrate to the developer and the customer that the software 
	meets its requirements.
	\item To discover faults or defects in the software where the behaviour of 
	the software is incorrect, undesirable or does not conform to its 
	specifications.
\end{enumerate}

\p The overall outcome of the testing process is to build confidence about the 
correctness of a software.\nl

\p According to \citet{PR00}, the objectives of software testing process are:
\begin{itemize}
\item Testing is a process of executing a program with the intent of finding an
error.
\item A good test case is one that has a high probability of finding an as-yet
undiscovered error.
\item A successful test is one that uncovers an as-yet-undiscovered error.
\end{itemize}

\p Software development process requires the initial test plan to be set ahead of
the actual business logic plan. The need for such plan is mandatory because it
gives the software engineer or the developer some clues of how the product 
conforms to standards. In addition, software extensibility is another issue that
software developers take into account. A developer has to conduct a series of 
test cases to make sure that the added component or extension has not affected 
the overall behaviour of any other component. Needless to say, test cases allow
the developer to experiment with many alternative designs as long as one of 
the available alternatives fits the most with the existing system. In addition,
fixing a bug\footnote{software bug: an error in a computer program or system.}
may introduce another bug or two. The question is: ``How can the developer be
sure that a change or upgrade is safe?'', and an answer to this question would
be by using test cases.\nl

\p \citet{PR05} suggests that a software bug occurs when one or more of the 
following five rules is true:
\begin{enumerate}
\item	The software doesn't do something that the product specification says it
		should do.
\item	The software does something that the product specification says it 
		shouldn't do.
\item	The software does something that the product specification doesn't 
		mention.
\item	The software doesn't do something that the product specification doesn't
		mention but should.
\item	The software is difficult to understand, hard to use, slow, or in the 
		software tester's eyes will be viewed by the end user as just plain not 
		right.
\end{enumerate}

\p \citet{PR00} suggests a set of testing principles to be taken in 
consideration while conducting a software development plan:
\begin{itemize}
	\item All tests should be traceable to customer requirements
	\item Tests should be planned long before testing begins
	\item The Pareto principle applies to software testing
	\item Testing should begin ``in the small'' and progress toward testing ``in
		  the large''
	\item Exhaustive testing is not possible
	\item To be most effective, testing should be conducted by an independent
			third party
\end{itemize}

\vfill

\p \citet{EG07} suggest four primary objectives of software testing: 
\begin{enumerate}
	\item Identify the magnitude and source of development risk reducible by
	testing
	\item Perform testing to reduce identified risk
	\item Know when testing is completed
	\item Manage testing as a standard project within the development project
\end{enumerate}

\begin{figure}[!htbp]
  \centering
  \fbox{
    \includegraphics[width=0.6\textwidth]{deq}
  }
  \caption{Regression of a polynomial function}
  \label{fig:deq}
\end{figure}

\p Testing is used either to remove as many bugs as possible before the final 
release of a software, or to check the integrity of the software after a change 
is made. To illustrate, developers may prefer dealing with as many bugs as 
possible rather than getting complaints from end users when a feature is found 
to be defective. Testing is aimed to narrow the gap between the result and 
actual result. An analogy to the rule of software testing is the regression 
analysis used in statistics, the more testing is performed, the closer the 
business logic to the real actual expected output, (Figure \ref{fig:deq}).
The more test cases performed on a feature, the more that feature is 
expected to behave properly and be error free. One of the very well-known unit 
testing tool available in the market is \ac{JUnit} \footnote{\ac{JUnit} is a 
test framework aimed to test, report and validate correctness of a software by 
using variety of assertions.}.\nl

\p One of the issues with using JUnit is that it needs to be written manually 
for every object behaviour. Therefore, a prior knowledge of the whole business
logic might be required. Needless to say, performing one assertion against every
behaviour tends not to be enough to detect extreme or critical situations. An 
analogy to mathematics would be testing the square root function against 
negative numbers, very large positive numbers and zero. Therefore, Junit4 has 
introduced a parameterized test which can solve the problem by feeding well 
known calculated results with their inputs and test them against the method 
that has just been programmed to ensure large coverage of as many extreme 
situations as possible. 

\subsection{Software Testing Methodologies}
\subsubsection{Design-Based Test Architecture}
The design based test architecture associates test procedures with the hardware 
and software design components of the system application, \citep{DE99}. Testing
at this level is mainly focused on the non-functional requirements of a software
design such as memory resources management, response time, connectivity, 
security, availability and performance.

\subsubsection{Technique-Based Test Architecture}
The technique-based test architecture associates the requirement for the 
procedures with the test techniques defined within the test program model,
\citep{DE99}. The technique-based test architecture provides the development 
team a clear picture of how the system should be tested by identifying 
requirements and the method for each requirement to be tested. 
\subsubsection{Test Models}

\p White-box testing
\footnote{
Some authors refer
to white-box testing as glass-box testing which indicates that the programmer
assess the internal structure of the given program, \citep{KC93}
}
: the tester has access to the internal structure of 
the entire packages and the referenced libraries. This method is used when the 
test coverage is important, \citep{DE99}. White box testing
uses several methods such as fault insertion, string test, error handling, 
statement coverage, decision coverage, condition coverage, path coverage, data 
flow coverage, memory leak and cyclomatic complexity.\nl

\begin{figure}[!htb]
  \centering
  \fbox{
    \includegraphics[width=0.5\textwidth]{bb-wb}
  }
  \caption{A comparison between white-box and black-box testing}
  \label{fig:bb-wb}
\end{figure}

\p Black-box testing: the tester does not have full access to the 
entire structure of the software as compared to white-box testing. 
Figure \ref{fig:bb-wb}, copied from \citep{PR05}, shows the difference 
between the two approaches. In other way, black-box testing is used 
when the internal implementation is not important and mainly to assure the 
correctness of the software, \citep{DE99}. The techniques used in black box 
testing are: boudary value analysis, cause-effect graphing, random testing, 
error guessing, regression testing, stress (volume) testing, replication 
testing, data integrity testing, backup and recoverability, configuration 
testing, performance testing, functional testing, security testing, equivalence 
partitioning, operational readiness testing, user acceptance testing, 
compatibility/conversion testing, benchmark testing, usability testing
and alpha/beta testing. \citet{GD04} points out some of the advantages 
and disadvantages of using black box test which is listed 
in Tabel \ref{table:bb-ad}.\nl

\begin{table}[!htb]
\begin{center}
	
  \begin{tabular}{ |l|p{5cm}|p{5cm}| }
  \hline
						& Advantages & Disadvantages \\ \hline \hline
     Black-box testing	& - Allows to carry out the majority of testing classes,
						    most of which can be implemented solely by black box
						    tests. 
						& - Possibility that coincidental aggregation of several
						    errors will produce the correct response for a test 
						    case, and prevent error detection.\\
						& - For testing classes that can be carried out by both 
						    white and black box tests, black box testing 
						    requires fewer resources than those required for 
						    white box testing of the same software package.
						& - Absence of control of line coverage. In cases where
						    black box testers wish to improve line coverage,
						    there is no easy way to specify the parameters of
						    the test cases required to improve coverage. \\
						&
						& - Impossibility of testing the quality of coding 
							and its strict adherence to the coding standard.\\
						    
	\hline					    
  \end{tabular}
\end{center}
	\caption{Advantages and disadvantages of black-box testing}
	\label{table:bb-ad}
\end{table}

\p Grey-box testing: refers to a class of testing techniques  that
combines black-box and white-box testing techniques, \citep{SE02}. \nl

\p Sand-box testing: is a method used to separate the actual development 
software from testing environment. An example of sandbox testing is version 
control or what is known in the software industry as \ac{SCM}
\footnote{\ac{SCM} : a software configuration management system maintains a 
history of changes done in the structure and contents of the managed project. 
This serves primarily the very precise goal of navigating to and retrieving a 
specific version in the project evolution, i.e. \ac{SVN} and \ac{CVS},
\citep{VL07}}. 
The SVN server is used to keep record of every revision made and a developer 
can check out the latest revision, makes the changes required, test it and 
validates correctness of the applied modifications and then checks it into the 
repository when required, \citep{EG07}.  Sub-versioning employs the agility of
a software in the since it provides the courage for a developer to make changes 
which is one of the basis of \ac{XP} principles, \citep{SV07}.
\citet{ML09} states that \ac{XP} is built on five basic principles which are:
(communication, simplicity, feedback, courage, respect). Out of these five
fundamentals, courage means that a developer is never afraid of making changes
to the code as long as there is a way to revert the work done, or throughly test
the new feature developed for defects that might have been introduced. \nl

\subsubsection{Test-Driven Development}
\p In test-driven development, the tests are the driving force of the whole 
process: not even a line of production code is supposed to be written without 
a test code, covering its functionality. In contrast with test-first discipline,
in test-last the program code is written first and unit tests are created 
afterwards. \citep{HE05}\nl

\p Test-driven development is an agile software development practice which 
is promoted by \ac{XP} and in the literature, sometimes it is called \ac{TF}.
\citet{ML09} claims that the key characteristic of \ac{TF} is that programmers
start writing test cases before the actual production code. This approach allows
development team to divide the work among them where some members can work on 
writing test cases while others exploring the requirements. Once a considerable 
amount of test cases are generated against what is so called mock implementation
\footnote{Mock objects: are production classes that have dummy implementation,
some developers start a project by designing a prototype that has no 
dynamic functionality and every object has its own behaviours and attributes
in a static state. After the progression of the project development, those mock
objects behaviours' are modified to reflect the real world requirements.}. 
\ac{TL}, on the other hand, is another approach used in software development
where test cases are generated after the production code is about to be released
to the clients to assess the correctness of program and verify if the 
requirements have been met or not.\nl
\p According to \citet{HE05}, \ac{TDD}  is believed to have several advantages:
\begin{itemize}
	\item tests provide instant feedback
	\item writing tests encourages the programmer to decompose the problem into 
		  manageable tasks (high granularity)
	\item frequent test runs ensure a certain level of quality
\end{itemize}
\p \citet{PM11} claim that the benefits of \ac{TDD} compared to iterative 
\ac{TL} is rather small and relatively unimportant, although \ac{TDD} effects 
are positive in general. \citet{MM02} investigate \ac{TDD} effectiveness and the
result of their experiment indicates that \ac{TDD} approach is more favoured  
than \ac{TL} approach by measuring the quality of code in both groups.

\subsubsection{Test Code Metrics}
The code quality may be considered as important as the final product itself. 
Although consumers are unaware of what is going behind the scene
but more interested in the output and usability of the final product, 
the quality of the code written could determine the survival of a software 
application when the requirements change. To assess production code quality,
some benchmarks are used to find dead code, unnecessary declarations
\footnote{Unnecessary declarations: the case when resources are declared
but never have been referenced in the program. Some programming languages, 
such as C enforces that a parameters can not be just declared if never 
referenced.\citep{DP12}}, or an improper logic that might lead to 
memory leak or result in unpleasant user experince\footnote{
Improper logic can be parts of code that will lead to potential bug such
as infinite loops trap, unsafe threading, or over consumption of 
hardware resources}.\nl

\subsubsection{Structural versus Functional Testing}
\p Structural testing is a white box testing. The main concern is proper 
selection of program or subprogram paths to exercise during the battery of 
tests. \nl

\p Functional testing is a black box testing where functions are tested by
feeding them inputs and examining the output. Internal program structure is 
rarely considered.\nl

\subsubsection{Automated Software Testing}
\p Automating the process of test generation is another approach used 
to push the development process and make developers concentrate on the
main goal which is producing the final output rather than spending
much of the effort on testing. In addition, the change of requirements
may lead to removal of some parts of the code, and this means that the
test cases associated with that removed code is deemed useless. There are
several tools capable of navigating code paths such as \ac{JPF},
which is developed by \ac{NASA} \citep{WV04} and 
Google \ac{CodeProAnalytix} \citep{GCP12}. These automated
test generation tools can provide test cases for simple mathematical  
operations which is not always the case when it comes to development.
Moreover, automated test tools lack an important aspect of testing, 
which is understanding the semantic and meaning of every behaviour.
For instance, a calculator class may contain several operations that
are related to do simple math such as (add, subtract, multiply, divide, etc.)
which have a clear semantic and the program can predict the behaviour 
of the operation. However, there are some cases where the inputs are 
complex and highly dependent on the logic used by the programmer. 


\subsection{Test Coverage and Software Testing Quality}
\p Having test cases passed is a thrilling experience for many developers
especially after a long day of debug and correct process. The important 
question to be asked is whether the test cases cover a large portion of the
code or not. Needless to say, test cases are not necessarily a prove of 
correctness or a 100\% guarantee that the code written is bullet-proof. 
Therefore, developers tend to generate as many test cases to cover
most of the scenarios.\nl

\p Test adequacy can be defined as a measure of a given test set designed to 
test a given program to determine whether or not the program meets the 
requirements, \citep{MA08}.\nl 

\subsubsection{The Notion of Test Adequacy}
\p \citet{HZ97} pointed out that defining the criterion of a test method shall
lead to an adequate testing. The following are some basic notions underlying 
adequacy criteria: \begin{itemize}
\item Statement coverage. In software testing practice, testers are often 
required to generate test cases to execute every statement in the program
at least once. 
\item Branch coverage. Similarly, the branch
coverage criterion requires that all control transfers in the program under test
are exercised during testing. The percentage of the control transfers executed 
during testing is a measurement of test adequacy.
\item Path coverage. The path coverage criterion requires that all the execution
paths from the program's entry to its exit are executed during testing.
\item Mutation adequacy. Software testing is often aimed at detecting faults in
software.
\end{itemize}

\p \citet{MA08} states that the criterion which has to be used to measure the 
adequacy of test cases as follows:\nl
\begin{quote}
	\begin{em}
		  ``A test T for program P with the set of requirements R is considered 
		  adquate if each path in P is traversed at least once''
	\end{em}.\nl
\end{quote}
\p The test coverage is a quality metric used to measure the distribution 
of tests performed on a software. Test coverage tools are used to find the 
parts of a software that are not tested and also can be used to identify 
methods that are not necessary since they were never been used by any object
(dead code\footnote{Dead code: is the part of code which is deemed 
unreachable either because of a condition which can never be satisfied
or server function is never called by any client.}). 
\ac{Emma} is a code coverage tool that can be used in conjunction with 
\ac{JUnit} to get a measure in percentage of how much has been tested in every 
component in a software.


\Section{Accomplished Project}
\subsection{Drivers and Motivation}
\p When the summer work started, test cases were generated manually. However, 
the time required to test a package is long and what made the process even 
slower was understanding the functionality of every class in each system. 
Nevertheless, the test coverage was not so promising indeed.
Therefore, a small dynamic unit test generation program (mini-jProbe) was
created. Although that small program did the job of creating test cases, a 
{\tt NullPointerException} used to be thrown almost in every class because of
calling uninitialized methods parameters. This has resulted in a lower test 
coverage since many methods just throw an {\tt IllegalArgumentException} if 
the arguments are checked and verified before any operation is performed, or a
{\tt NullPointerException} is thrown when a parameter is referenced by a 
method.\nl

\p Mini-jProbe was designed to intentionally pass null pointer to any 
reference type parameter. The reason for not initializing parameters is not a
straightforward statement to be written. In fact, developing jProbe was 
motivated mainly because of this feature. The main issue is that reference 
objects can be initialized by any of its declared public constructors, or the
empty constructor in case no constructor is declared. The problem is which 
constructor should be used if an object has more than one constructor block? 
In addition, some constructors require some parameters to be passed as
arguments. Dealing with primitives tends not to be an issue, however; reference
objects are delat with the same way as the target object. This could lead into a
recursion loop that has no end if there is a circular dependency.\nl

\p To sum up, creating an initialized copy of an object may require initializing
it, and to initialize it, some arguments have to be passed, the argument
type can be a primitive, or reference. Reference types can be an abstract
or an interface or abstract class\footnote{interfaces are contracts that
defines the structure of the object without actual implementation, while
abstract classes are those classes which has some methods (behaviours) declared
abstract to be left to the concrete implementation to define the action taken
depending on the context.}. There is a need for a project that analyses the code
and determines how to initialize reference type objects accordingly.
 
\subsection{Problem Space}
\p As mentioned earlier in the previous section, the task was to generate test 
cases for different systems under the Qualitas Corpus set of projects. Some of 
the systems in the Qualitas collection has its test cases, however; test
coverage  is not as high as expected. The main requirement is to automate test 
cases generation for each system.  
\subsection{Project Specifications}


\subsubsection{Alternatives}
The reader of this report might wonder: ``why would not using an existing test 
generation tool from those available already in the software design community 
instead of re-inventing the wheel?''. The answer to this question is summarized 
in the following points:
\begin{itemize}
	\item Test generation tools such as \ac{CodeProAnalytix} is an Eclipse 
	\ac{IDE} plugin and can generate test cases and provides test coverage as
	well. However, it allows for generating test cases on a class by class basis.
	Using this tool for the Apache Ant project might require spending the whole
	summer.
        \item In addition to the previoius note, \ac{CodeProAnalytix} may not
        be able to generate test cases for objects that requires another objects
        to be initilaized as a parameter. Instead a failed test case is generated, 
        but work very well with primitive data types parameter which is not
        always the case \footnote{data types in Java are of two types, 
        primitives and reference. Primitives are those data types which needs
        no initialization such as int, double, char, byte, long, float, short 
        and boolean. Reference data types, on the other hand, are objects and 
        needs to be initialized before they are used.}.
        \item Other options have been explored such as using paid test
        generation tools, however, there might be a need to learn how to use 
        them. Moreover, using a paid test generation tool that has not been 
        used before may have two risks:
		\begin{enumerate}
			\item The tool may not meet the expectations
		 	\item Learning a new tool requires time, try and error, which might
			 lead of abandoning the tool and losing its value.
		\end{enumerate}
	\item The tools explored are capable of visiting public members of classes only
\end{itemize}
\subsubsection{Requirements and Conceptual Analysis}
\p The main requirement is to produce a tool that can help generate test cases for 
any java project. In addition, the tool has to generate test cases for each and every
class in the project. Moreover, the tool is supposed to provide coverage indication
of the tests generated and the success rate.\nl

\p {\tt Package and classpath exploring:} the developed project should have a
feature of exploring packages and each class under every package. The feature
should also include exploring the native java classpath since objects are
most likely to use the objects defined by java language which is common to
all systems.\nl

\p {\tt Class visiting:} the test generation tool is expected to traverse each method
and member declared in the class. This includes public, private, protected, default access,
and package access members.\nl

\p {\tt Bean inspection:} bean\footnote{
{\bf JavaBeans} a Java Bean is a
reusable software component that can be visually manipulated in builder tools
and conforms to certain features which are (introspection, customization,
events, properties, and persistence, \citep{OR10}).
} inspection feature is responsible for accessing an initialized object
properties by reading its getters. Bean inspection is also supposed to run
mutator\footnote{a mutator method (sometimes called setter) is a method used to
control changes to a variable, \citep{WP12}} methods and then inspect the effect
of such method on the attributes of the object. Deep inspection is a feature
required especially for arrays, and child objects that can be inspected as
well.\nl

\p {\tt Object initialization:} the test generation tool has to have some
intelligence of initilaizing objects on the fly as they are required to 
initialize other objects or to be used in calling methods declared in 
objects.\nl

\p {\tt Code generation:} this feature is required to write 
reference object initializers on the fly, compile them and get the returned
value. Moreover, the same code used to initialize an object or a parameter used
as an argument to initialize an object is used in the test case code
generation. In addition, the code generation feature is responsible for
producing unit test cases against every class in the system.\nl
 
\p {\tt Reporting test coverage:} the tool has to show the percentage of test
cover and the success rate of the testability of the given system.

\subsection{Development}
\p jProbe is an open-source project hosted on Google Code server and can be
accessed from this link: \url{http://code.google.com/p/jprobe/}, (Figure
\ref{fig:svn}).
The reason for choosing this repository server was to share the project among the development
team. In addition, issue tracking is integrated with the \ac{SVN} control
system.
By the end of the practicum period, there were 22 issues, 12 issues are solved
and the other are still under investigation, (Figure \ref{fig:issues}). 


\begin{figure}[htbp]
  \centering
      \includegraphics[width=0.8\textwidth]{svn}
   \caption{jProbe project home page on Google Code}
  \label{fig:svn}
\end{figure}


\subsubsection{Project structure}
jProbe is organized into packages according to the features that the project is
expected to perform. Therefore there are 9 packages (Figure \ref{fig:packages})
in the main source directory and the following is the explaination for each package:
\begin{itemize}
  \item {\tt org.jprobe} this package defines the contracts of a java project
  that is going to be used for test cases generation.
  \item {\tt org.jprobe.classpath} contains specifications and implementation of
  how the classpath of a java project is explored and also used to load classes
  as necessary.
  \item {\tt org.jprobe.codegen} is responsible for java code generation which
  is either to initialize an object or to generate test cases.
  \item {\tt org.jprobe.dummy} contains dummy classes which are used mostly in
  testing the accuracy of detecting initializers and exploration of an object
  specializations and generalizations. This package is not a
  feature, but it can be considered as an accessory package.
  \item {\tt org.jprobe.instantiate} this package provides tools that are used
  to make an object instantiated (initialized).
  \item {\tt org.jprobe.ui} a \ac{GUI} for the project, (Figure \ref{fig:gui})
  \item {\tt org.jprobe.utils} a utility tools such as java tools compilers, ant
  script build, file \ac{I/O} operations, \ac{RegEx} parser, and \ac{XML}
  operation tools.
\end{itemize}

	\begin{figure}[htbp]
	  \centering
	  \fbox{
	    \includegraphics{packages}
	  }
	  \caption{jProbe project packages}  \label{fig:packages}
	\end{figure}

\subsubsection{How to use jProbe}
The following code snipest provide an example on how to use jProbe.  

\begin{Verbatim}[frame=single]
 String dirPath = "./jmoney-0.4.4/"; //provide the project directory
 File directory = new File(dirPath);
 JavaProjectInterface javaProject = new JavaProject(directory);
 JUnitGeneratorAdaptor jga = new JUnitGeneratorAdaptor(javaProject);
 jga.generateAll();  //this method is the one generating 
                     //test cases against every class in the project
\end{Verbatim}

\begin{figure}[htbp]
  \centering
  
    \includegraphics[width=0.85\textwidth]{gui}
  
  \caption{jProbe \ac{GUI}}
  \label{fig:gui}
\end{figure}

 \begin{figure}[htbp]
  \centering
  \fbox{
    \includegraphics[width=0.75\textwidth]{issues}
  }
  \caption{Current open issues (jprobe project issue traking)}
  \label{fig:issues}
\end{figure}

 
\Section{Conclusion}

\subsection{Results}
\p On one hand, mini-jProbe can generate 100\% test coverage on the class
level, and the maximum line coverage was about 35\%. However, with hundreds of
errors and generated test cases need to be manually modified to make them run.
Moreover, the success rate for the test cases generated at most 30\%\nl

\p jProbe, on the other hand, can generate 100\% class level coverage, and up to
50\% line coverage. The generated test cases code may contain some errors,
however, they are minimal. The success rate for jProbe reaches up to 70\% in the
best case senarios.\nl

\p It has to be noted that some systems when tested using mini-jProbe and
jProbe, the results where almost the same. However, jProbe provided a more
meaningful test cases.\nl

\p Class dependency might not be directly related to the context of this
project. However, some classes were found in some project with cyclic dependency
which resulted in making one of the classes uninitializable and therefore
automating the  generation of test cases may not be possible.\nl 

\subsection{Recommendations and Future Work}
\p The work on the jProbe project is incomplete, a proof of concept has not yet
been established. Therefore, it is recommended that the organization carries out
the development of this project.\nl 

\subsection{Personal Reflection}

\p The overall impression about the organization's training program  
was an excellent experience. Many new skills were gained, including 
interpersonal skills, communication with superiors, and meeting 
deadlines by delivering the deliverables. Although, the final 
developed system was partially complete due to time constraint, the 
whole training program has been an excellent opportunity to 
get a hands-on experience about how actual businesses operate. 
Being involved in a team has also provided the skill to be a 
team player and interact with other departments.    

\newpage
\bibliographystyle{apacite} 
%\bibliographystyle{plainnat}
\bibliography{references}
\end{document}